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5 INTERACTION MODALITIES FOR MULTIMEDIA DELIVERY AND 

PRESENTATION 

This application claims priority to Provisional Application No. 60/052,576, filed 
July 15, 1997. 

10 

FIELD OF THE INVENTION 

The present invention relates to transferring information over a network. More 
particularly, the present invention relates to systems and methods for improving data 
flow based on user preferences. 

BACKGROUND OF THE INVENTION 

Remote interactivity is an important issue in both multimedia research and 
standards development. In the known art, when an encoder transmits multimedia data 
to a decoder, the status of the transmission can be determined at the decoder and then 
20 fed back, through a back channel, to the encoder. 

Historically, many people were convinced that using a backchannel with real- 
time multimedia was either impractical or impossible. The concept of a back channel 
is known in the art. Currently, broad status reports are sent from a receiver to a 
transmitter. These status reports provide very general information to the transmitter, 
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5 such as whether a terminal is receiving a transmission at all. Detailed information 
regarding how a transmission is received, or how a transmission is displayed, is 
typically not fed back to the transmission source. And typically, if this information is 
fed back to the transmission source, the transmitter may adjust the transmission or may 
not, but does so without regard to the receiver's individual preferences. 

10 

SUMMARY OF THE INVENTION 

The prior art lacks a meaningful way of passing suggestions from the receiver 
to the sender to take certain actions to improve the quality of what is displayed based 
on the receiver's preferences. 

15 Asa context for the present invention, but not the only context, consider a 

server transmitting multimedia data to a client. It would be beneficial for the client to 
have some way of feeding information back to the server to maximize the quality of the 
multimedia transmission. For example, although not the only example, if a client 
desires to receive multimedia data from one or more servers, and the client is also 

20 running applications that consume memory and negatively impact on the way the 
multimedia data is displayed, the client would benefit from being able to suggest to the 
server various ways to alter the transmission such that the client's display is improved. 
Some examples of suggestions to the server, although not the only examples, include 
suggestions to cut out certain transmitted frames and reduce the transmitted audio 

25 channels. 

With this contextual example in mind, in one embodiment of the present 
invention, data is received and its quality of service is analyzed. Then, based on this 
analysis and on received user preferences, the receiver formulates a suggestion for the 
encoder to alter the characteristics of the transmitted data. Finally, the receiver 
30 transmits this suggestion to the transmitter. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a system overview of an embodiment of the present invention. 
FIGURE 2 is a flow chart illustrating an embodiment of a method according to 
35 the present invention. 
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5 FIGURE 3 is a flow chart illustrating another embodiment of a method 

according to the present invention. 

FIGURE 4 is a pictorial representation of an apparatus according to one 
embodiment of the present invention. 

FIGURE 5 is a pictorial representation of another apparatus according to one 
1 0 embodiment of the present invention. 

DETAILED DESCRIPTION 

The present invention relates to methods and systems for improving the quality 
of displayed data that has been transmitted over a network. For the purposes of the 
15 present invention, the word quality refers to the audio and visual characteristics of 
displayed data. 

Referring now in detail to the drawings, FIGURE 1 illustrates a system 
overview of an embodiment of the present invention. As a context for the present 
invention, but not the only context, consider Sender 1 08 to be a multimedia data server 

20 that transmits high-quality multimedia data to a client. The client can use a 
backchannel to interact with the server, analyzing a set of parameters that represent the 
quality of the transmitted data, comparing the quality to a set of user preferences, and 
formulating a set of suggestions that are fed back to the server to alter the data such that 
the quality of the displayed data is affected. 

25 In one embodiment of the present invention, as displayed in FIGURE 1 , sender 

108 transmits data to a receiver 101 which then displays the data. At 102, receiver 101 
analyzes the quality of the display. At 103, a set of user preferences is received, and 
at 104, the quality analysis is compared to the user preferences. At 105, suggested 
actions are formulated and fed back along the backchannel to sender 1 08. When sender 

30 108 receives the suggestions, sender 108 checks its status 106 and matches its current 
status with a set of possible actions 107. An action is chosen and the transmitted data 
is altered accordingly. 

FIGURE 2 is a flow chart illustrating an embodiment of a method according to 
the present invention. In this embodiment, at step 20 1 , received data is displayed at the 

35 receiver. For the purposes of this invention, the word "received" includes, but is not 
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5 limited to, "created," "generated," and "selected." Therefore, the displayed data in step 
201 may be received from an outside source, but it may also be, for example, internally 
generated. In one embodiment of the present invention, the displayed data is 
audiovisual, or multimedia, data. 

At step 202, user preferences are received. In one embodiment of the present 
10 invention, the user preferences can specify preferences to improve the quality of the 
display. For example, the user preference can include data that says, in effect, "when 
I am busy, degrade audio, degrade video, and drop texture." In one embodiment of the 
present invention, the user preferences can be altered by the user during the 
transmission. In another embodiment of the present invention, the user preferences are 
1 5 fixed after the transmission begins. 

At step 203, a set of parameters that represent the quality of the displayed data 

V- 

is analyzed. In one embodiment of the present invention, the set of parameters includes 
system load. For example, but not the only example, assuming a seven-bit word, zero 
can represent no load, and one hundred twenty seven can represent the system being 

20 completely busy. In another embodiment of the present invention, the set of parameters 
-includes component load, where a component is chosen from the set comprising the 
central-processing unit (CPU), the graphics card, and the texture-mapping engine. 

At step 204, a suggestion is formulated for the sender to alter the characteristics 
of the transmitted data. The suggestion is formulated by comparing the results of the 

25 analysis of the displayed data to the user preferences. The user preferences set some 
threshold for various parameters, and if the results of the analysis falls outside of that 
threshold, the receiver suggests to the sender to change various parameters of the 
transmitted data. 

In one embodiment of the present invention, the formulated suggestion includes 
30 timing information identifying the point in time where the data was collected, and 
timing information identifying the point in time when the suggested action should be 
honored. 

In another embodiment of the present invention, the formulated suggestion 
includes a suggestion to alter the rate at which frames are transmitted for display. In 
35 another embodiment of the present invention, the formulated suggestion includes 
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5 altering the color depth. In another embodiment of the present invention, the 
formulated suggestion includes a suggestion to alter the window size. For the purposes 
of the present invention, "window size" means how much screen size area the scene 
covers. As an example, but not the only example, assume that the displayed scene 
covers the entire screen, but because of limited memory, the quality of the display is 

1 0 degraded. The formulated suggestion can include a suggestion to reduce the size of the 
display, thereby decreasing the need for computing power at the receiver and increasing 
the quality of the display. 

In another embodiment of the present invention, the formulated suggestion 
includes a suggestion to alter audio channel characteristics. For the purposes of the 

15 present invention, the phrase "audio channel characteristics" includes bandwidth, 
number of audio channels, and spatial audio capabilities. As an example, but not the 
only example, assume that the quality of the display is degraded due to memory 
constraints at the receiver. The formulated suggestion can then include a suggestion 
that the sender decrease the number of audio channels broadcast, thereby decreasing the 

20 need for computing power at the receiver and increasing the quality of the display. 

In another embodiment of the present invention, the formulated suggestion 
includes a suggestion to alter the graphics hardware load. For the purposes of the 
present invention, the phrase "graphics hardware load" includes, but is not limited to, 
matrix multiplication, texture mapping, and texture mapping memory. 

25 In another embodiment of the present invention, the formulated suggestion 

includes a suggestion to alter the CPU load or the RAM amount available. 

FIGURE 3 is a flow chart illustrating another embodiment of the present 
invention. At 301, data is sent by a sender to a receiver. In one embodiment of the 
present invention, the data sent at 301 is audiovisual, or multimedia data. In response 

30 to this sent data, the sender receives a suggestion at 302 to alter the transmitted data. 
In one embodiment of the present invention, the received suggestion includes timing 
information identifying the point in time where the data was collected, and timing 
information identifying the point in time when the suggested action should be honored. 
In another embodiment of the present invention, the received suggestion 

35 includes a suggestion to alter the rate at which frames are transmitted for display. In 
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5 another embodiment of the present invention, the received suggestion includes a 
suggestion to alter the color depth. In another embodiment of the present invention, the 
received suggestion includes a suggestion to alter the window size. As an example, but 
not the only example, assume that the displayed scene covers the entire screen, but 
because of limited memory, the quality of the display is degraded. The received 

1 0 suggestion can include a suggestion to reduce the size of the display, thereby decreasing 
the need for computing power at the receiver and increasing the quality of the display. 

In another embodiment of the present invention, the received suggestion 
includes a suggestion to alter audio channel characteristics. As an example, but not the 
only example, assume that the quality of the display is degraded due to memory 

1 5 constraints at the receiver. The received suggestion can then include a suggestion that 
the sender decrease the number of audio channels broadcast, thereby decreasing the 
need for computing power at the receiver and increasing the quality of the display. 

In another embodiment of the present invention, the received suggestion 
includes a suggestion to alter the graphics hardware load. In another embodiment of 

20 the present invention, the received suggestion includes a suggestion to alter the CPU 
load or the RAM amount available. 

At step 303, the status is checked, and based on the current status and the 
received suggestion, at step 304, an action is selected from a set of actions, and the 
transmitted data is altered accordingly. 

25 FIGURE 4 is a pictorial representation of an apparatus according to one 

embodiment of the present invention. The apparatus includes a port 40 1 coupled to 
processor 402, and memory 403, also coupled to processor 402 and to port 401 . In one 
embodiment of the present invention, memory 403 stores instructions adapted to be 
executed by the processor to display received data, analyze the quality of the displayed 

30 data, formulate a suggestion for the encoder to alter the characteristics of data to be 
received, and then send, through port 401 , the formulated suggestion to alter the quality 
of the received data. 

For the purposes of this application, memory includes any medium capable of 
storing instructions adapted to be executed by a processor. Some examples of such 

35 media include, but are not limited to, floppy disks, CDROM, magnetic tape, hard 
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7 



5 drives, and any other device that can store digital information. In one embodiment, the 
instructions are stored on the medium in a compressed and/or encrypted format. As 
used herein, the phrase "adapted to be executed by a processor" is meant to encompass 
instructions stored in a compressed and/or encrypted format, as well as instructions that 
have to be compiled or installed by an installer before being executed by the processor. 

10 In one embodiment of the present invention, the memory stores instructions 

adapted to be run on the processor to receive a user preference to be used in the 
analysis. In another embodiment of the present invention, the formulated suggestion 
includes timing information identifying when the data was collected, and timing 
information identifying when the suggested action should be honored. 

15 In another embodiment of the present invention, the formulated suggestion 

includes a suggestion to alter the rate at which frames are transmitted for display. In 
another embodiment of the present invention, the received suggestion includes a 
suggestion to alter the color depth. In another embodiment of the present invention, the 
received suggestion includes a suggestion to alter the window size. As an example, but 

20 not the only example, assume that the displayed scene covers the entire screen, but 
because of limited memory, the quality of the display is degraded. The formulated 
suggestion can include a suggestion to reduce the size of the display, thereby decreasing 
the need for computing power at the receiver and increasing the quality of the display. 
In another embodiment of the present invention, the formulated suggestion 

25 . includes a suggestion to alter audio channel characteristics. As an example, but not the 
only example, assume that the quality of the display is degraded due to memory 
constraints at the receiver. The formulated suggestion can then include a suggestion 
that the sender decrease the number of audio channels broadcast, thereby decreasing the 
need for computing power at the receiver and increasing the quality of the display. 

30 In another embodiment of the present invention, the formulated suggestion 

includes a suggestion to alter the graphics hardware load. In another embodiment of 
the present invention, the formulated suggestion includes a suggestion to alter the CPU 
load or the RAM amount available. 

FIGURE 5 is a pictorial representation of an embodiment of the present 

35 invention. In one embodiment of the present invention, port 501 is coupled to 
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5 processor 502, and memory 503 is coupled to port 501 and processor 502. Memory 503 
contains transmission instructions adapted to be run on the processor to transmit data 
to a receiver, receive a suggestion to alter the transmitted data, and then decide to alter 
the transmitted data. 

In one embodiment of the present invention, the received suggestion includes 

10 timing information identifying the point in time where the data was collected, and 
timing information identifying the point in time when the suggested action should be 
honored. In another embodiment of the present invention, the received suggestion 
includes a suggestion to alter the rate at which frames are transmitted for display. In 
another embodiment of the present invention, the received suggestion includes a 

1 5 suggestion to alter the color depth. In another embodiment of the present invention, the 
received suggestion includes a suggestion to alter the window size. As an example, but 
not the only example, assume that the displayed scene covers the entire screen, but 
because of limited memory, the quality of the display is degraded. The received 
suggestion can include a suggestion to reduce the size of the display, thereby decreasing 

20 the need for computing power at the receiver and increasing the quality of the display. 

In another embodiment of the present invention, the received suggestion 
includes a suggestion to alter audio channel characteristics. As an example, but not the 
only example, assume that the quality of the display is degraded due to memory 
constraints at the receiver. The received suggestion can then include a suggestion that 

25 the sender decrease the number of audio channels broadcast, thereby decreasing the 
need for computing power at the receiver and increasing the quality of the display. 

In another embodiment of the present invention, the received suggestion 
includes a suggestion to alter the graphics hardware load. In another embodiment of 
the present invention, the received suggestion includes a suggestion to alter the CPU 

30 load or the RAM amount available. 

The present invention has been described in terms of several embodiments 
solely for the purpose of illustration. Persons skilled in the art will recognize from this 
description that the invention is not limited to the embodiments described, but may be 
practiced with modifications and alterations limited only by the spirit and scope of the 

35 appended claims. 
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